In general, the present invention relates to computer storage systems that use multiple disk drives combined into an array of disk drives that xe2x80x98appearsxe2x80x99 to a host computer as a single logical storage unit that includes fault-tolerance characteristics and yields better overall performance. It has been found that this type of disk array can be made more fault-tolerant by redundantly storing information/data in various ways over the multiple drives, including any of the established RAID (which stands for Redundant Array of Independent Disks) architectures/levels 0 through 5, and subsequent variations/hybrids thereof, whether mapped as hardware- or software-based RAID. More particularly, the invention relates to a new system and unique method for migrating information/data stored on a source volume into a RAID structured assemblage of data carriers whereby a separate back-up disk (or other external storage media) is not needed and the source data carrier can become part of the RAID structured assemblage.
A storage medium/media can be any data carrier or recording medium into/onto which information (such as data) can be read/copied, such as magnetic (diskettes, hard disks, Iomega""s ZIP(copyright)/JAZ(copyright)/Click!(trademark) disks, tapes, drums, core, thin-film, etc.), optic (CD-ROM, CD-E, CD-R, CD-RW, DVD, and other devices whereby readout is with a light-source and photodetector), and magneto-optic media (media for which optical properties can be changed by an applied magnetic fieldxe2x80x94used in high end drives). Before data can be written to a storage medium it must be xe2x80x98formattedxe2x80x99 (organized and prepared to receive data). In the case of a formatting a disk, a file system is generally xe2x80x98burned-inxe2x80x99. Many file system types are currently in use, such as: FAT/FAT16/FAT32 (File Allocation Table) for DOS/Windows 95/98(copyright) PC""s, HPFS (High Performance File System) used with OS/2 server disks, NTFS (NT File System) used with Windows NT(copyright), ISO 9660 for CD-ROMs, ISO 13346 for DVDs, UDF (Universal Disk Format) for bigger-capacity disks like DVD RAM, Novell""s NetWare(copyright) server operating system has its own proprietary file system, and UNIX-based servers have their own filing system.
The file system on a disk, for example, operates as an interface between the operating system and the drive in which the disk is located. When a software application, such as MS Word, is asked by a PC user to read a file from a disk, the operating system on that computer engages the file system to open the file. The file name can include a xe2x80x98drive letterxe2x80x99 for the drive associated with the disk on which the chosen file is located. Since the file system includes information about physical location of individual files on the hard drive, the file system is engaged to find the relevant physical location on the hard drive, reads the data there, then delivers it to the operating system. During formatting, a disk (such as the hard disk found in a PC""s hard drive, Iomega Corporation""s portable ZIP(copyright)/JAZ(copyright)/Click!(trademark) disks, and diskettes) is divided into multi-byte sectors (512-bytes, for example) distributed along concentric xe2x80x98tracksxe2x80x99. A hard disk formatted using FAT, for example, has its administrative data stored on an outer track sector(s). Each hard disk, and if partitioned each partition of the disk, often contains four areas in FAT (this depends upon the Partition Type): (1) a boot record, generally in the first sector, (2) FAT administrative data (usually two identical areas), (3) the root directory, and (4) the data area where file information and subdirectories, beyond the root directory, are gathered in clusters (cluster size depends on disk size) locatable by the file system for reading. The formatting of a CD-ROM and a DVD is different as these disks have one-long spiral track divided into identifiable sectors.
A physical hard disk can be divided into different partitions, each of which (for example, in systems running Windows 95(copyright) and Windows NT(copyright)) can be treated as logically-separate drives. For example, such partitioning of a PC hard drive is usually done by a program called FDISK. A boot record is found on every hard disk (regardless of its specific file system) and contains information about the disk or its partitions. Boot record information enables the file system to handle the disk and also includes a small program used during system start-up. The root directory of a FAT partition is a listing of files and other directories on the respective partition/disk. The root directory of a hard disk with a FAT system, has 512 file or directory xe2x80x98entrancesxe2x80x99 and is unique from other directories in that it is of a fixed size and is located in the same physical location.
When a hard disk has been partitioned, thus creating each partition into a xe2x80x98logical drivexe2x80x99 as perceived by the computer""s operating system, the file system must be given information allowing it to recognize this partitioning. Information about the physical location of the beginning and the end of each partition, called the Master Boot Record (MBR)xe2x80x94because it describes the entire hard disk, is stored in the first sector of the physical hard disk. Then, at the beginning of each partition (or, logical drive) is a boot record describing the partition that follows.
Once a computer has been powered-up and its start-up program has finished POST (Power On Self Test) and the loading of BIOS (Basic Input/Output System) routines, the boot process begins. First, the MBR is read and the sector number of the primary partition""s boot record is located. Any hard disk that has been partitioned has a primary partition (for example, a drive-letter C: can be assigned) from which booting of the computer takes place and the operating system is read (the small boot program in the primary partition is engaged to activate the loading of the operating system filesxe2x80x94which are stored, in a DOS-based PC, as hidden system files in the primary partition).
In the late-1980""s, a paper entitled xe2x80x9cA Case for Redundant Arrays of Inexpensive Disks (RAID)xe2x80x9d was published by authors from the University of California Berkeley, that described various types of disk arrays. The basic idea of RAID was to combine multiple small, what they deemed xe2x80x98inexpensivexe2x80x99 disk drives into an array of disk drives which yields performance exceeding that of a Single Large Expensive Drive (SLED) and is perceived by a host computer as a single logical storage unit or drive. RAID disk arrays, which are interface-independent, are made fault-tolerant by redundantly storing information in various ways. Five types of array architectures, RAID-1 through RAID-5, were originally defined in this paper, each providing disk fault-tolerance. Another level, RAID-0, has emerged which is more-correctly a xe2x80x98non-redundantxe2x80x99 array of disk drives.
Fundamental to RAID is xe2x80x9cstripingxe2x80x9d of magnetic media, which is a method of concatenating multiple drives into one logical storage unit by partitioning the storage space of each drive in the array into strips (or stripes) which may be as small as one sector of the disk (512 bytes) or as large as several megabytes. These strips are then interleaved round-robin, so that the combined space is composed alternately of strips from each drive. In effect, the storage space of the drive array is shuffled like a deck of cards. Often, the type of application environment in which the RAID is used, I/O or data intensive, will determine whether large or small strips are used.
RAID Level 0, which is technically not redundant, has data that is split across drivesxe2x80x94thus, it""s commonly referred to as xe2x80x9cstripingxe2x80x9dxe2x80x94resulting in higher data throughput. Since no redundant information is stored, performance is very good, but the failure of any disk in the array results in data loss. RAID Level 1 provides redundancy by writing all data to two or more drivesxe2x80x94thus, it""s commonly referred to as xe2x80x9cmirroringxe2x80x9d. Since one drive is used to store a duplicate of the data, the cost per megabyte is high. The performance of a level 1 array tends to be faster on reads and slower on writes compared to a single drive, but if either drive fails, no data is lost. RAID Level 2, which uses Hamming error correction codes, is intended for use with drives which do not have built-in error detection. All SCSI drives support built-in error detection, so this level is of little use when using SCSI drives (see SCSI explanation below). RAID Level 3 is similar to Level 4 except that Level 3 stripes data at a byte level across several drives, with parity stored on one drive (byte-level striping requires hardware support for efficient use). RAID Level 4 stripes data at a block level across several drives, with parity stored on one drive (this parity information allows recovery from the failure of any single drive). The performance of a Level 4 array is very good for reading data, or xe2x80x9creadsxe2x80x9d (as is true for Level 0). Writing data (xe2x80x9cwritesxe2x80x9d), however, require that parity data be updated each time. Because only one drive in the RAID 4 array stores redundant data, the cost per megabyte of a Level 4 array can be fairly low. RAID Level 5 is similar to Level 4, but distributes parity among the drives (RAID-5 is popular). Because parity data must be skipped on each drive during reads, however, the performance for reads tends to be considerably lower than a Level 4 array.
A xe2x80x9chardware RAIDxe2x80x9d based system manages the RAID subsystem array independently from the host computer and presents to the host computer only a single logical disk per RAID array. Hardware RAID requires that each subsystem have a dedicated RAID controller. A SCSI controller-based hardware solution allows the controller to span the RAID subsystem over multiple SCSI channels. An external SCSI hardware solution moves the RAID handling xe2x80x9cintelligencexe2x80x9d into a controller that is located as part of the external RAID subsystem.
Just like any other computer software application, a xe2x80x9csoftware-RAIDxe2x80x9d array occupies host system memory and is host operating system dependentxe2x80x94therefore, the performance of a software-based array directly depends upon server CPU performance and load. Because software-RAID competes with other applications running concurrently on a host CPU, software-based arrays may degrade overall server performance. Hardware-RAID arrays do not occupy any host system memory, and hardware-RAID does not depend on the host operating system. Thus, except for the array functionality, hardware-based RAID schemes have very little in common with software-based implementations. Hardware-RAID arrays are highly fault tolerant and the array logic is based in hardware (software is not required to boot). A RAID-array implemented in software can only be functional when the array software has been read from the disks and is memory-resident. Software-based implementations commonly require a separate boot drive, not included in the array.
The widely-used small computer system interface (SCSI) protocol was developed for industry groups, under the American National Standards Institute (ANSI) and International Standards Organization (ISO) guidelines, to provide an efficient peer-topeer I/O bus. Devices that conform with the mechanical, electrical, timing, and protocol requirements (including the physical attributes of I/O buses used to interconnect computers and peripheral devices) of the SCSI parallel interface will inter-operate. This allows several different peripherals (hard disk drives, removable disk drives, tape drives, CD-ROM drives, printers, scanners, optical media drives, and so on) to be added at the same time to a host computer without requiring modifications to the generic system hardware. The working draft of the SCSI Parallel Interface-2 Standard (SPI-2), as modified (Rev. 16, dated Oct. 14, 1997), defines the cables, connectors, signals, transceivers, and protocol used to interconnect SCSI devices. The SPI-2 working draft states that a SCSI bus consists of all the conductors and connectors required to attain signal line continuity between every driver, receiver, and terminator for each signal. In operation, a SCSI bus is a bidirectional, multimaster bus which can accommodate peer to peer communications among multiple computer processing units (CPUs) and multiple peripheral devices. A SCSI device is a device (any computer peripheral) containing at least one SCSI port and the means to connect the drivers and receivers to the bus.
A SCSI primary bus is one that provides for and carries 8-bit or 16-bit data transfer. A SCSI secondary bus carries an additional 16-bit data bus that, when used in conjunction with a 16-bit primary bus, provides for a 32-bit data transfer path (although the latter is not, yet, widely used). SCSI devices may connect to a bus via 8-bit, 16-bit, or 32-bit ports. To date, SCSI parallel interface devices may be implemented with either 50, 68, or 80 pin connectors (whether shielded or unshielded). Eight-bit devices are referred to as xe2x80x9cnarrowxe2x80x9d devices (carried by 50 pin cables and connectors) and devices which permit parallel transfer of 16-bits or 32-bits at a time are called xe2x80x9cwidexe2x80x9d devices (carried by cables and connectors with at least 68 pin). Narrow and wide, as used in connection with SCSI compliant systems, refers to width of the data path. Data travels fast over SCSI buses, with Ultra SCSI data transfer speeds up to 40 MBps, wide Ultra2 LVD up to 80 MBps, and wide Ultra3 up to 160 MBps. As it is well known, a typical data transfer operation over a SCSI bus between a SCSI controller (or xe2x80x9chost adapterxe2x80x9d) located in a host computer system, to a target device (such as a disk drive) has seven SCSI xe2x80x9cphasesxe2x80x9d: (1) ARBITRATION, (2) SELECTION, (3) RESELECTION, (4) COMMAND, (5) DATA, (6) STATUS and (7) MESSAGE. For example, during the COMMAND phase, a SCSI command is transferred from the host adapter to a target (drive); and so on. Host adapter functional circuitry is typically maintained on a host bus adapter (HBA) chip on a printed circuit board structure referred to as a host adapter board (HAB) for connection to a PC host via an expansion slot. California based Adaptec, Inc., as well as the assignee hereof, design and distribute host adapters for making narrow to wide SCSI connections.
Transceivers transmit and receive electrical signals/information on a SCSI bus using single-ended (SE) signal transmission mode protocol or a differential protocol (either high voltage differential, HVD, or low voltage differential, LVD). Since the SE and HVD alternatives are mutually exclusive and the LVD and HVD alternatives are mutually exclusive, a bus can support only all HVD devices or SE/LVD signal transmission devices. In addition to SE, LVD, HVD transceivers and terminators, SE/LVD multimode transceiver and multimode terminator architecture has been designed for operation over a SCSI bus in the event it""s expected that a combination of SE and LVD protocol devices will be combined thereon.
Fibre Channel is a newer interface technology (emerging along with Serial Storage Architecture (SSA) and IEEE P1394) capable of transferring data as fast and faster than an Ultra3 SCSI system can, over fiber optic cabling as well as copper transmission media. Fiber Channel-type host bus adapters are installed into a host computer expansion slot just as SCSI host bus adapters are installed. Fibre channel connections are often associated with the term xe2x80x9cloopxe2x80x9d (from the name Fibre Channel arbitrated loop) rather than xe2x80x9cbusxe2x80x9d (as SCSI devices are connected). There are actually other types of Fibre Channel connections, called xe2x80x9cpoint to pointxe2x80x9d and xe2x80x9cfabric.xe2x80x9d With fibre channel, communication between hosts and devices does not have to be done directly. Instead, users can employ hubs and switches between devices on the Fibre Channel network. Hubs and switches can be used to create Fibre Channel xe2x80x9cstorage networksxe2x80x9d. Fibre Channel cabling can be copper (can be up to 30 meters in length) or fiber optic (currently up to 10 Km). In addition, no termination is necessary for fibre channel as is required in SCSI. Fiber optic ports directly placed on a peripheral device allow only for connections to fiber optic cabling. Commercially available adapters exist that allow a SCSI-compliant device to be connected to a Fiber Channel loop.
As shown in FIG. 1, the conventional method by which a xe2x80x98genericxe2x80x99 non-RAID structured hard disk 10 (regardless of the type of interface protocol used to communicate therewith) can be xe2x80x98convertedxe2x80x99 into a RAID volume or a RAID-structured disk array can be copied into a different RAID-structured array is by utilizing a large, external separate back-up device 12 in communication with a processor and associated computer memory. The processor (often, the source volume host processor) of device 12 is engaged to read the data originally stored on the source disk 10 into its memory, then structure the data according to the desired RAID Level/mapping and download it to newly-assembled disks (16, 17, 18,19) to create the new/separate RAID array 14. The additional time, effort and the expense of needing, only temporarily for the RAID conversion, a separate high-end back-up device 12, makes this conventional RAID conversion method undesirable as a mechanism by which to convert non-RAID storage media into a RAID volume on a regular basis. The conventional method is simply not a good RAID structure solution for customers of workstations, servers, mainframes, high-end PC""s, and so on.
Therefore, a new useful mechanism is needed by which an initially non-RAID storage volume can be converted into a RAID array, and can preferably also convert a storage volume structured in an initial RAID level into a different RAID level, without the need to employ a costly external back-up device (and its associated interconnect hardware). Without reasonable, accurate, and cost-effective solutions at hand for structuring a storage volume into a desired RAID structure in a timely manner, it is very difficult for computer information technologists/specialists to convert a single hard disk, for example, with valuable data into a RAID structure (with its important redundancy/data protection features). As anyone who has tried to recover lost data on a failed disk will readily understand: certain circumstances make it imperative that a RAID array be available for storing valuable data.
It is a primary object of this invention to provide a system for automatically migrating a portion of a collection of information located on one or more source data carriers (also xe2x80x98source volumexe2x80x99) into an assemblage of data carriers that includes at least a first storage medium and can include the source data carrier. Information from the source data carrier is read into a computer memory, preferably in strips (as small as a single bit, several bytes, or as large a xe2x80x98blockxe2x80x99 of information as one chooses) and written to the first (second, third, fourth, fifth, and so on) storage medium in a predetermined structure (e.g., into a RAID structure). A remaining portion of the collection of information (this remaining portion can comprise one or more remaining-sub-portions of the strip(s)) is written to the source data carrier in conformity with the predetermined structure. It is a further object to provide an assemblage of data carriers comprising a first storage medium and a source data carrier; the first storage medium to have a portion of a collection of information originating as a first structure on the source data carrier, and the source data carrier to contain a remaining portion of the first structure collection reordered in conformity with a RAID structure. It is also an object of this invention to provide a method of automatically migrating a portion of a collection of information located on one or more source data carriers to an assemblage of data carriers-once a first sub-portion of a first strip has been written to a first storage medium of the assemblage according to a predetermined structure, a remaining portion of the collection is written to the source carrier so it can become part of the assemblage.
The innovative assemblage of RAID structured data carriers, and associated system and method of automatically migrating a portion of a collection of information located on a source data carrier, as contemplated and described herein, can accommodate: many different storage volumes (having a single or a multitude of individual storage carriers); a wide variety of types of computer interface protocols; and many different types of computerized devices/systems. Furthermore, the simple, effective design allows the system, method, and program code of the invention to be tailored-to, readily installed, and run using currently-available processors and memory.
Unlike the conventional systems currently in use, the innovative assemblage of data carriers, and associated system and method, of the invention utilize the source data carrier on which the data to be converted is initially found to make the source carrier part of the assemblage as structured. None of the currently-available systems take advantage of the availability of the source data carrier. In the spirit of this design, one, two, three, and so on, storage media can be included in the initial source storage volume and/or incorporated in the structured assemblage of data as will be further appreciated.
The advantages of providing the flexible new system, the assemblage of RAID structured data carriers, and associated new method of automatically migrating a portion of a collection of information, as described herein, follow:
(a) Cost Reductionxe2x80x94A separate back-up disk (or other external storage media) is not needed to convert a non-RAID structured storage volume (such as a workstation or server hard drive) into a RAID structured storage volume/array, or to convert a RAID structured storage volume into a different RAID Level structured storage volume/array; plus one less storage medium is needed in the newly structured assemblage of data carriers since the original/source data carrier becomes part of the new assemblage.
(b) Design Flexibility and Versatilityxe2x80x94Due to the many different configurations, including the several popular RAID level/mappings (each of which has data-redundancy benefits plus potential as well as real drawbacks), into which data can be structured on an external storage medium (whether the xe2x80x98storage volumexe2x80x99 is encased within a computer housing or accessed as a stand-alone unit), system engineers and information technology specialists of many different computer platforms can readily use the more-flexible data structure conversion solutions offered by the instant invention. The entire non-RAID hard disk of a workstation(s), server(s), PC, etc., or individual partitions of the disk, file systems thereon, and so on, can be converted into a chosen RAID level set. Furthermore, by allowing one the option of including the source data carrier in the final assemblage of carriers the method and system provide even more operational flexibility.
(c) The system, assemblage of data carriers, and associated method aid in reducing inadvertent data loss on a source volume, such as a workstation or server hard disk, by allowing the source volume to be readily converted, in an automatic fashion, into a RAID structured volume (or from one RAID structure volume to another RAID structure). Unlike any conventional process, this powerful novel system and method allows one to optionally include the source in the final structured volume.
(d) Design simplicityxe2x80x94Simplifying the design reduces overall system costs, such as the cost of keeping on-hand a temporary operational back-up tape or disk system and its cabling plus highly-skilled labor and supplier-support costs associated with correctly setting-up/wiring, running, then disassembling the back-up system. Reducing system costs, in-turn reduces the cost to perform the important information-structure conversion of source storage volumes (e.g., from a non-RAID storage volume to a RAID-mapped volume). The instant invention allows for automatic conversion without needing an especially-large memory buffer through which the information is migrated.
(e) Process simplificationxe2x80x94The system and method migrate the collection of information of a source data carrier independently of the source carrier""s file system and the way the source is originally logically partitioned. Creating a newly-structured volume according to the invention (in the form of an assemblage of data carriers) does not require that specific information in partitions of the source data carrier, such as boot record(s), root directories and associated files, be interpreted during the process.
Briefly described, again, the invention includes a system for automatically migrating a portion of a collection of information located on a source data carrier(s) into an assemblage of data carriers. The system has a computer memory for temporary storage of a first strip of the collection; the assemblage of data carriers has at least a first storage medium to which at least a first sub-portion of the first strip is written from computer memory; and once the portion of the collection has been so migrated according to a predetermined structure, at least a remaining portion of the collection can be written to the source data carrier and ordered in conformity with the predetermined structure (such that the assemblage now further includes at least one of the source data carriers).
Additional further distinguishing features are readily apparent: The collection of information can be initially non-RAID structured or RAID structured, and the structure of the assemblage can be RAID structured (different than the initial structure). The first strip can comprise: a first sub-portion targeted for migration to the first storage medium; a second sub-portion migrated to a second storage medium; a third sub-portion migrated to a third storage medium, and so on for each newly-added storage media of the assemblage; as well as a remaining sub-portion (as part of the remaining portion that gets written back to the source data carrier). Alternatively, the entire first strip (having no information from the remaining portion targeted for the source data carrier), depending on its size, can be written to the first storage medium, a second strip of the collection written to a second storage medium, a third strip of the collection written to a third storage medium, and so on. Physical location information about data written to the assemblage can be added to the storage media or source data carrier. Furthermore, a user interface can be included for entering an input to optionally instruct that a remaining portion of the collection be migrated back to the source data carrier or to at least the first storage medium. One might wish to exclude an obsolete drive from the assemblage.
Also characterized is an assemblage of data carriers having a first storage medium and a source data carrier; at least the first storage medium to contain a portion of a collection of information originating as a first structure on the source data carrier, this portion having been migrated through a computer memory in communication with the source data carrier and written as RAID structured. The source data carrier of the assemblage contains a remaining portion of the first structured collection reordered in conformity with the RAID structure. The information written to the storage medium and source data carrier is, preferably, verified before it leaves computer memory.
Also characterized herein is an associated method of automatically migrating a portion of a collection of information located on source data carrier(s) to an assemblage of data carriers. This method comprises the steps of reading into a computer memory, a first strip of the collection; and once a first sub-portion of the first strip has been written to a first storage medium of the assemblage according to a predetermined structure, writing a remaining portion of the collection to the source data carrier such that it becomes part of the assemblage. The method may operate on the collection of information on the source data carrier whether in a partition-by-partition fashion or independently of any initial logical partitioning of the source storage volume by finding strips of a chosen size via a logical block address, etc. Additional, further distinguishing associated features of the method can be readily appreciated as set forth herein.
Further characterized is a computer executable program code on a computer readable storage medium. The program code has a first program sub-code for reading into a computer memory, a first strip of a collection of information located on a source data carrier, and writing a first sub-portion of said first strip to a first storage medium of an assemblage of data carriers according to a predetermined structure; and a second program sub-code for writing a remaining portion of the collection to the source data carrier. Code for verifying information written to the assemblage can be added.